Add prompt section to game configs - #26
Conversation
|
We don't have information indicating how many turns an agent has left. I will add this. I think this should be another prompt field that then gets inserted somewhere in the turn templates of mini-SWE-agent. |
I think we can do that already with the |
|
ah nvm, we only have the model config, not its stats at the moment. I'll add that to mini in the future |
|
Yeah exactly. Maybe this is something that's better to just build into mini directly? For this PR, I won't worry about it then. |
| rounds: int | ||
| working_dir: str | ||
|
|
||
| def render_and_set_prompts(self): |
There was a problem hiding this comment.
Oh I see, this is because we need to render variables within the game description?
But wouldn't the clean way then simply be to have a render_prompts method that gets called when we use this object in the agent?
Why do we assign it with setattr and then later delete attrs in to_dict etc.? That feels pretty unnatural and confusing. Normally, we just want to track state in the object (without any duplication), and then either use properties or methods to transform it (but maybe I'm missing something)
There was a problem hiding this comment.
ah ok yeah this could be cleaner. But basically
- we define a set of prompts in CodeClash configs
- and then we define a set of prompts in Mini configs
The Mini config will reference ({{...}}) a prompt defined in the CodeClash config. But then the CodeClash config will reference the game state, and it needs to be updated as the game proceeds (e.g. round).
So then if i do Template(Mini template).render(CodeClash template), it won't populate the CodeClash template correctly with the metadata.
So this extra function is essentially to populate the CodeClash template first. Basically, this was the approach I came up to handle render a template, that is then rendered in another template.
There could be a more elegant way to do this. I was going for the simplest, most flexible solution. I think it's ok, but open to changing this if there's something obviously easier?
There was a problem hiding this comment.
Totally happy to just merge this and if I can come up with something more elegant, I can just create a PR or something? I was thinking about this yesterday a little bit and I think there might be slightly more elegant ways to do this, but I need to experiment a bit (it would be equivalent)
There was a problem hiding this comment.
Sg sg yeah like we discussed in meeting - I'll resolve ur comments right now 🙏🏼 and then just merge, so we can just move on for now.
2 participants
This PR introduces the
promptssection of a CodeClash config. The prompts contain free form text + instructions that can + should be incorporated by whatever agentic editing scaffold (in our case,mini-swe-agent) is being used to facilitate an LM's ability to edit a codebase.To standardize the game information that is available to an agent, this PR also replaces the
template_varsvariable with aGameContextdataclass that also facilitates the re-rendering of prompts as the game state changes across multiple rounds.